DTE AICCOMAS 2025

Student

Generating Digital Twins of Multibody Systems Using Motion Capture, IMU Sensing, and an LLM

  • Wang, Shu (University of Wisconsin-Madison)
  • Wang, Jingquan (University of Wisconsin-Madison)
  • Wu, Jinlong (University of Wisconsin-Madison)
  • Serban, Radu (University of Wisconsin-Madison)
  • Negrut, Dan (University of Wisconsin-Madison)

Please login to view abstract download link

We present a method for generating digital twins of multibody systems by utilizing inertial measurement units (IMU) sensors, two motion capture (mocap) markers, and a Large Language Model (LLM). Traditional approaches to inferring rigid body dynamics often depend on computer vision to capture position and orientation data, which can be difficult to apply in systems where internal mechanical structures are obscured, inaccessible, or experience occlusion. The proposed approach circumvents these challenges by employing IMU sensors (one per system body) and two mocap markers to capture the precise motion trajectories of the system’s components. By combining collected trajectory data with the multi-body dynamic differential-algebraic equations (DAEs) of motion, we infer the joint parameters between mechanical bodies, which allows us to identify the model parameters associated with the physical system. These parameters are subsequently provided to an LLM, which produces a Chrono rendition of the physical system. Subsequent simulations to analyse the dynamics of the system can be carried out in PyChrono. This method offers improvements in flexibility and robustness compared to traditional video-based techniques, making it useful in modeling complex or constrained mechanical systems such as those found in robotics, automotive systems, and industrial machinery. Ultimately, the integration of IMU and mocap data with advanced dynamic modeling expands the potential for accurate and real-time digital twin creation, facilitating more effective system diagnostics, optimization, and decision-making.